Skip to content

Conversation

@karlseguin
Copy link
Collaborator

When V8 calls the ResolveModuleCallback that we give it, it passes the specifier which is essentially the string given to from:

import {x} from './blah.js';

We were taking that specifier and giving it to the page. The page knew the currently executing script, an thus could resolve the full URL. Given the full URL, it could either return the JS content from its module cache or fetch the source.

At best though, this isn't efficient. If two files import the same module, yes we cache the src, but we still ask v8 to re-compile it. At worse, it crashes due to resource exhaustion in the case of cyclical dependencies.

ResolveModuleCallback should instead detect that it has already loaded the module and return the previously loaded module. Essentially, we shouldn't be caching the JavaScript source, we should be caching the v8 module.

However, in order to do this, we need more than the specifier, which might only be a relative path (and thus isn't unique). So, in addition to a module cache, we now also maintain an module identifier lookup. Given a module, we can get its full path. Thankfully ResolveModuleCallback gives us the referring module, so we can look up that modules URL, stitch it to the specifier, and get the full url (the unique identifier) within the JS runtime.

Need more real world testing, and a fully working example before I celebrate, but for sites with many import, this appears to improve performance by many orders of magnitude.

When V8 calls the ResolveModuleCallback that we give it, it passes the specifier
which is essentially the string given to `from`:

```
import {x} from './blah.js';
```

We were taking that specifier and giving it to the page. The page knew the
currently executing script, an thus could resolve the full URL. Given the full
URL, it could either return the JS content from its module cache or fetch
the source.

At best though, this isn't efficient. If two files import the same module, yes
we cache the src, but we still ask v8 to re-compile it. At worse, it crashes
due to resource exhaustion in the case of cyclical dependencies.

ResolveModuleCallback should instead detect that it has already loaded the
module and return the previously loaded module. Essentially, we shouldn't be
caching the JavaScript source, we should be caching the v8 module.

However, in order to do this, we need more than the specifier, which might only
be a relative path (and thus isn't unique). So, in addition to a module cache,
we now also maintain an module identifier lookup. Given a module, we can get
its full path. Thankfully ResolveModuleCallback gives us the referring module,
so we can look up that modules URL, stitch it to the specifier, and get the
full url (the unique identifier) within the JS runtime.

Need more real world testing, and a fully working example before I celebrate,
but for sites with many import, this appears to improve performance by many
orders of magnitude.
@karlseguin
Copy link
Collaborator Author

@karlseguin karlseguin merged commit bbafb04 into main Jun 23, 2025
11 checks passed
@karlseguin karlseguin deleted the module_loading branch June 23, 2025 00:54
@github-actions github-actions bot locked and limited conversation to collaborators Jun 23, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants